Skip to content

[ISSUEE 60] Add features to the broker and nameserver - #61

Open
linjiemiao wants to merge 1 commit into
apache:masterfrom
silotrd:issue-60-dev
Open

[ISSUEE 60] Add features to the broker and nameserver#61
linjiemiao wants to merge 1 commit into
apache:masterfrom
silotrd:issue-60-dev

Conversation

@linjiemiao

Copy link
Copy Markdown
  1. add affinity / securityContext / imagePullSecrets / tolerations / nodeSelector / podAnnotations / priorityClassName

   nodeSelector / podAnnotations / priorityClassName
Comment on lines +407 to +412
Affinity: broker.Spec.Affinity,
SecurityContext: broker.Spec.SecurityContext,
ImagePullSecrets: broker.Spec.ImagePullSecrets,
Tolerations: broker.Spec.Tolerations,
NodeSelector: broker.Spec.NodeSelector,
PriorityClassName: broker.Spec.PriorityClassName,

@AdheipSingh AdheipSingh Nov 15, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to write functions for each of them and handle nil value scenarios, by directly adding this you are forcing the user at all times to specify these values, the operator will crash if you don't pass in tolerations.
@liuruiyiyang since these values will go to both broker/nameserver we can create single functions which can be used both in broker and nameserver

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 This pr added features that help enrich operator, hope that corresponding null check would be supplemented.

Comment on lines +324 to +329
Affinity: nameService.Spec.Affinity,
SecurityContext: nameService.Spec.SecurityContext,
ImagePullSecrets: nameService.Spec.ImagePullSecrets,
Tolerations: nameService.Spec.Tolerations,
NodeSelector: nameService.Spec.NodeSelector,
PriorityClassName: nameService.Spec.PriorityClassName,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, we need to write functions to get values to handle nil/empty scenarios

Comment on lines +407 to +412
Affinity: broker.Spec.Affinity,
SecurityContext: broker.Spec.SecurityContext,
ImagePullSecrets: broker.Spec.ImagePullSecrets,
Tolerations: broker.Spec.Tolerations,
NodeSelector: broker.Spec.NodeSelector,
PriorityClassName: broker.Spec.PriorityClassName,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 This pr added features that help enrich operator, hope that corresponding null check would be supplemented.

Comment on lines +61 to +65
NodeSelector map[string]string `json:"nodeSelector,omitempty"`
// PodAnnotations you can use annotations to attach arbitrary non-identifying metadata to objects.
PodAnnotations map[string]string `json:"podAnnotations,omitempty"`
// PriorityClassName defines priority class's name
PriorityClassName string `json:"priorityClassName,omitempty"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems null checks are required for them, would define them as pointer be better?

Comment on lines +68 to +72
NodeSelector map[string]string `json:"nodeSelector,omitempty"`
// PodAnnotations you can use annotations to attach arbitrary non-identifying metadata to objects.
PodAnnotations map[string]string `json:"podAnnotations,omitempty"`
// PriorityClassName defines priority class's name
PriorityClassName string `json:"priorityClassName,omitempty"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems null checks are required for them, would define them as pointer be better?

@RockteMQ-AI

Copy link
Copy Markdown

⚠️ Merge conflict detected

This PR has conflicts with the base branch and cannot be merged. Please rebase or merge the base branch into your branch and resolve the conflicts:

git fetch origin
git checkout issue-60-dev
git rebase origin/main
# resolve conflicts, then:
git push --force-with-lease

This is a one-time reminder. Feel free to @mention me for a re-review after conflicts are resolved.


Automated notification by github-manager-bot

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR modifies 7 file(s) with 351 lines of diff. No test changes detected — consider adding test coverage.


Automated review by github-manager-bot

@@ -56,6 +56,21 @@ type BrokerSpec struct {
VolumeClaimTemplates []corev1.PersistentVolumeClaim `json:"volumeClaimTemplates"`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test changes detected alongside source modifications. Consider adding tests to cover the changes.

@RockteMQ-AI

Copy link
Copy Markdown

This PR has been open since September 2020 (nearly 6 years) and currently has merge conflicts with the master branch.

Status check: Is this PR still actively being worked on? The feature additions (affinity, tolerations, nodeSelector, etc.) are valuable, but the code needs to be rebased to resolve conflicts.

If this is still relevant, please:

  1. Rebase onto the latest master
  2. Address the changes requested by @caigy
  3. Update the PR description with current status

If this has been abandoned, consider closing it.


Automated review by github-manager-bot

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

PR received and logged for review. This PR requires detailed code review by a maintainer.

Diff size: 351 lines
Author: linjiemiao (NONE)


Automated review by RockteMQ-AI

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Review of PR #61: [ISSUEE 60] Add features to the broker and nameserver

Findings: 4 issue(s) identified (1 critical).
CLA: unknown

Please address the inline comments above.


Automated review by github-manager-bot

VolumeClaimTemplates []corev1.PersistentVolumeClaim `json:"volumeClaimTemplates"`
// The name of pod where the metadata from
ScalePodName string `json:"scalePodName"`
// Affinity, affinity and anti-affinity scheduling

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New pointer/slice/map fields (Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PodAnnotations) were added to BrokerSpec, but the PR does not include updates to zz_generated_deepcopy.go. If the generated DeepCopyInto is not regenerated, these fields are shallow-copied: Affinity/SecurityContext pointers and NodeSelector/PodAnnotations maps will be shared between the informer cache and the object handed to the reconciler. Any mutation corrupts the shared cache and can cause cross-reconcile state corruption. Run the operator-sdk/k8s codegen to regenerate deepcopy for both broker_types.go and nameservice_types.go.

Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: ls,
Annotations: broker.Spec.PodAnnotations,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new pod-scheduling fields are wired into getBrokerStatefulSet (Annotations, Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PriorityClassName) and similarly into nameservice_controller.go, but no test changes accompany this PR. There is no coverage verifying that CR-spec values propagate to the rendered StatefulSet PodTemplateSpec, nor that nil/empty values (e.g. unset Affinity, empty PriorityClassName) render safely. Add reconciler tests asserting these fields pass through correctly.

prometheus.io/scrape: "true"
securityContext:
allowPrivilegeEscalation: true
runAsUser: 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NameService securityContext sets allowPrivilegeEscalation: true, runAsUser: 0, runAsGroup: 0. This example is intended as a copyable template, and running the name server as root with privilege escalation enabled is a poor security pattern for an Apache-shipped reference. Consider a non-root runAsUser (the alpine image supports a dedicated UID) and allowPrivilegeEscalation: false, or at minimum document the risk and make it opt-in.

items:
type: object
type: array
affinity:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new complex Kubernetes types (affinity, securityContext, tolerations, imagePullSecrets items) are declared as bare type: object/item type: object with no OpenAPI v3 properties. This provides no server-side validation, so malformed scheduling rules are accepted silently and only surface as Pod creation errors later. For a cleaner CRD (and better kubectl explain output), define the full schema via x-kubernetes-preserve-unknown-fields or the concrete property trees. Same applies to rocketmq_v1alpha1_nameservice_crd.yaml.

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The PR correctly adds standard Kubernetes scheduling fields to both CRDs and controllers with consistent Go types, but lacks CRD schema validation, has a security-concerning example, and misses test coverage and annotation merge logic.

Findings

  • [WARNING] deploy/crds/rocketmq_v1alpha1_broker_crd.yaml:84 — The affinity field is declared as type: object with no structural schema. This allows any arbitrary YAML to pass CRD validation, deferring errors to the kube-scheduler at runtime. Consider using x-kubernetes-preserve-unknown-fields: true with a more specific schema, or at minimum document the expected structure. Same issue applies to securityContext, nodeSelector, and podAnnotations (all type: object), and to imagePullSecrets/tolerations (items: type: object). While this pattern is common in early-stage operators, it provides zero admission-time validation.
  • [WARNING] deploy/crds/rocketmq_v1alpha1_nameservice_crd.yaml:68 — Same unstructured schema issue as the broker CRD — affinity, securityContext, nodeSelector, podAnnotations all accept arbitrary objects. Since these fields are duplicated across two CRDs, consider extracting a shared validation schema (or a shared struct with code-generated OpenAPI) to keep them in sync and avoid drift.
  • [WARNING] example/rocketmq_v1alpha1_rocketmq_feature_cluster.yaml:128 — The example sets allowPrivilegeEscalation: true and runAsUser: 0 / runAsGroup: 0, which runs the NameService container as root with privilege escalation enabled. This is a security anti-pattern and should not be in an example file that users will copy. If root is truly required for RocketMQ, add a comment explaining why and recommend dropping to a non-root user where possible. At minimum, set allowPrivilegeEscalation: false.
  • [WARNING] pkg/controller/broker/broker_controller.go:404Annotations: broker.Spec.PodAnnotations directly assigns the spec map to the pod template. If a user later removes all annotations from the spec (setting it to nil or {}), the existing annotations on the live StatefulSet's pod template won't be cleared during an update unless the controller explicitly handles this in its update/reconciliation path. Verify that the update logic in updateBrokerStatefulSet (or equivalent) properly propagates annotation removal, not just addition.
  • [INFO] pkg/controller/broker/broker_controller.go:405 — All six new PodSpec fields (Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PriorityClassName) are assigned directly from the CR spec with no nil-guard. This is safe for the Kubernetes API (nil/empty values are valid), but it means any change to these fields will trigger a StatefulSet rolling update. For fields like nodeSelector or tolerations that an operator might want to change frequently, consider documenting this behavior so users understand the pod churn implications.
  • [WARNING] pkg/controller/nameservice/nameservice_controller.go:319 — Same annotation propagation concern as the broker controller: Annotations: nameService.Spec.PodAnnotations is a direct assignment. If the existing StatefulSet's pod template has annotations set by a mutating webhook or by a previous spec version, they will be silently overwritten on reconciliation. Consider merging annotations (existing + spec) rather than replacing, or document that spec annotations are authoritative.
  • [INFO] pkg/apis/rocketmq/v1alpha1/broker_types.go:72 — No test coverage is added for the new fields. The broker and nameservice controller tests should verify that: (1) a CR with these fields produces a StatefulSet with the correct PodSpec, (2) nil/empty values produce the same StatefulSet as before (backward compat), and (3) updates to these fields trigger a rolling update. Without tests, regressions in the reconciliation logic are likely to go undetected.

Automated review by github-manager-bot

type: array
affinity:
description: Affinity and anti-affinity scheduling
type: object

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The affinity field is declared as type: object with no structural schema. This allows any arbitrary YAML to pass CRD validation, deferring errors to the kube-scheduler at runtime. Consider using x-kubernetes-preserve-unknown-fields: true with a more specific schema, or at minimum document the expected structure. Same issue applies to securityContext, nodeSelector, and podAnnotations (all type: object), and to imagePullSecrets/tolerations (items: type: object). While this pattern is common in early-stage operators, it provides zero admission-time validation.

type: array
affinity:
description: Affinity and anti-affinity scheduling
type: object

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same unstructured schema issue as the broker CRD — affinity, securityContext, nodeSelector, podAnnotations all accept arbitrary objects. Since these fields are duplicated across two CRDs, consider extracting a shared validation schema (or a shared struct with code-generated OpenAPI) to keep them in sync and avoid drift.

- azure-k8s-2
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example sets allowPrivilegeEscalation: true and runAsUser: 0 / runAsGroup: 0, which runs the NameService container as root with privilege escalation enabled. This is a security anti-pattern and should not be in an example file that users will copy. If root is truly required for RocketMQ, add a comment explaining why and recommend dropping to a non-root user where possible. At minimum, set allowPrivilegeEscalation: false.

Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: ls,
Annotations: broker.Spec.PodAnnotations,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Annotations: broker.Spec.PodAnnotations directly assigns the spec map to the pod template. If a user later removes all annotations from the spec (setting it to nil or {}), the existing annotations on the live StatefulSet's pod template won't be cleared during an update unless the controller explicitly handles this in its update/reconciliation path. Verify that the update logic in updateBrokerStatefulSet (or equivalent) properly propagates annotation removal, not just addition.

ObjectMeta: metav1.ObjectMeta{
Labels: ls,
Annotations: broker.Spec.PodAnnotations,
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All six new PodSpec fields (Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PriorityClassName) are assigned directly from the CR spec with no nil-guard. This is safe for the Kubernetes API (nil/empty values are valid), but it means any change to these fields will trigger a StatefulSet rolling update. For fields like nodeSelector or tolerations that an operator might want to change frequently, consider documenting this behavior so users understand the pod churn implications.

Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: ls,
Annotations: nameService.Spec.PodAnnotations,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same annotation propagation concern as the broker controller: Annotations: nameService.Spec.PodAnnotations is a direct assignment. If the existing StatefulSet's pod template has annotations set by a mutating webhook or by a previous spec version, they will be silently overwritten on reconciliation. Consider merging annotations (existing + spec) rather than replacing, or document that spec annotations are authoritative.

// PodAnnotations you can use annotations to attach arbitrary non-identifying metadata to objects.
PodAnnotations map[string]string `json:"podAnnotations,omitempty"`
// PriorityClassName defines priority class's name
PriorityClassName string `json:"priorityClassName,omitempty"`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test coverage is added for the new fields. The broker and nameservice controller tests should verify that: (1) a CR with these fields produces a StatefulSet with the correct PodSpec, (2) nil/empty values produce the same StatefulSet as before (backward compat), and (3) updates to these fields trigger a rolling update. Without tests, regressions in the reconciliation logic are likely to go undetected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants